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This is in response to the appeal brief filed 25 March 2011 appealing from the Office 
action mailed 27 October 2010. 

(2) Related Appeals and Interferences 

The examiner is not aware of any related appeals, interferences, or judicial 
proceedings which will directly affect or be directly affected by or have a bearing on 
the Board's decision in the pending appeal. 

(3) Status of Claims 

The following is a list of claims that are rejected and pending in the 
application: 

Claims 1-3, 5, 7-15, 17, and 19-25 1 . 

(4) Status of Amendments After Final 

The examiner has no comment on the appellant's statement of the status of 
amendments after final rejection contained in the brief. 

(5) Summary of Claimed Subject Matter 

The examiner has no comment on the summary of claimed subject matter 
contained in the brief. 



1 Appellant states in pages 2 and "i" that claims "26-29" have been cancelled. Claim 30 is also 
cancelled. 
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(6) Grounds of Rejection to be Reviewed on Appeal 

The examiner has no comment on the appellant's statement of the grounds of 
rejection to be reviewed on appeal. Every ground of rejection set forth in the Office 
action from which the appeal is taken (as modified by any advisory actions) is being 
maintained by the examiner except for the grounds of rejection (if any) listed under 
the subheading "WITHDRAWN REJECTIONS." New grounds of rejection (if any) 
are provided under the subheading "NEW GROUNDS OF REJECTION". 

(7) Claims Appendix 

The examiner has no comment on the copy of the appealed claims contained 
in the Appendix to the appellant's brief. 

(8) Evidence Relied Upon 

• P. De Smet & D. De Vleeschauwer, "Motion-Based Segmentation using a 
thresholded merging strategy on watershed segments", 2 Proc. 1997 Int'l 
Conf. on Image Processing (ICIP ^97) pp. 490-493 (Oct. 1997). 

• B. Zhang, "^Harmonic Means - A Data Clustering Algorithm", Hewlett- 
Packard Laboratories Technical Report 1999-124 (Oct. 1999). 

• S. Herrman, H. Mooshofer, H. Dietrich, & W. Stechele, "a Video 
Segmentation Algorithm for Hierarchical Object Represenations and Its 
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Implementation", 9 IEEE Trans, on Circuits & Sys. for Video Tech. 1204-- 
1215 (Dec. 1999). 

(9) Grounds of Rejection 

The following ground(s) of rejection are applicable to the appealed claims: 

Claim Rejections - 35 USC § 103 

1. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 

obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as 
set forth in section 102 of this title, if the differences between the subject matter sought to be 
patented and the prior art are such that the subject matter as a whole would have been obvious at 
the time the invention was made to a person having ordinary skill in the art to which said subject 
matter pertains. Patentability shall not be negatived by the manner in which the invention was 
made. 

2. Claims 1-3, 5, 7, 10-15, 17, 19, and 22-25 are rejected under 35 U.S.C. 
103(a) as being unpatentable over "Motion-based Segmentation Using a 
Thresholded Merging Strategy on Watershed Segments" ("de_Smet") in view of "K- 
Harmonic Means-A Data Clustering Algorithm" (" Zhang" ). De Smet discloses using 
an iterative segment-merging technique to determine information for an image 
(abstract). 

Regarding claims 1, 13, and 25, in de Smet , an initial motion field is first 
determined with a block-matching technique on 4 x 4 blocks. De Smet . § 2.1. These 
initial block motion vectors are used for the initial segmentation. Id. at § 2.3. 
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Then, the step of performing the block-based motion estimation is the claimed step 
of "providing data points", and the motion vectors themselves in the initial motion 
field are the claimed "motion paths". Next, the segments are iteratively merged 
according to similar or shared motion, according to the K- means clustering 
algorithm. IcL This is the claimed step of "cluster[ing] the data points". When this 
process is finished, the result is a series of large segments corresponding to distinct 
moving regions of an image, each with an associated motion vector. IcL This is the 
claimed step of "providing motion estimation". However, the present invention 
specifies performing regression clustering according to a K-Harmonic Means 
function, which is not the same as the K- means function of de Smet . 

Zhang discloses the K-Harmonic Means data clustering algorithm. 
Regarding claim 1, 3, and 25, Zhang teaches selecting K centers m(l) from N data 
points x(i) (pg. 1), initializing center points (pg. 2) and performing an initial 
iteration (pg. 5), and calculating distance d(i,l) between data point x(i) and center 
point m(l) (pg. 4), calculating membership probability p(i,x) based on the distance 
via parameters q(i,k) and q(i) (pg. 5), and stopping when the recursively-calculated 
performance value stabilizes, that is, when its change with each iteration becomes 
small (pg. 5). Then, the K calculations of centers m(l) are the claimed "regression 
functions" for performing regression clustering according to the K-harmonic means 
function. The calculation of each iteration of the recursive function is the claimed 
recalculation based on the membership probability, as m(k) is dependent on 
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probability function p(i,k). The stabilization is the claimed "stopping criterion". In 
the particular application of "data compression and vector quantization", these 
center points are the claimed "motion paths" or motion vectors of de Smet , and 
distance d(i,l) is the claimed "error". 

De Smet discloses the claimed invention, except for using K-Harmonic Means 
function to perform regression clustering. Zhang teaches that it was known to 
perform data clustering with the K-Harmonic means function. Therefore, it would 
have been obvious to one having ordinary skill in the art at the time the invention 
was made to perform clustering based on a K-Harmonic means function, as taught 
by Zhang , rather than a linear function such as the K- means function of de Smet , 
since Zhang states in the abstract that K-harmonic clustering is less sensitive to 
detrimental effects from sub-optimal initialization than conventional clustering 
techniques. Additionally, the K-harmonic means function was specifically designed 
as an improved version of the K-means function, which "significantly improves the 
quality of clustering results" compared with K-means (abstract). Then, the de Smet 
algorithm contains a basic method that the present invention is an improvement 
thereof. The prior art Zhang describes a known technique, the K-Harmonic means 
clustering algorithm, applicable to the base segmentation method of de Smet , by 
substituting it for the K-means algorithm. Then, one having ordinary skill in the 
art would have recognized that applying the K-harmonic means algorithm of Zhang 
to de Smet would have yielded the predictable result of "significantly [improved] 
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quality of clustering results" and resulted in an improved system. Therefore, it is 
respectfully submitted that the use of K-harmonic clustering in de Smet is 
considered obvious, since it has been held that applying a known technique to a 
known method ready for improvement to yield predictable results involves only 
routine skill in the art. Dann v. Johnston , 425 U.S. 219, 230, 189 U.S.P.Q. 257, 261 
(1976); In re Nillsen . 851 F.2d 1401, 1403, 7 U.S.P.Q.2d 1500, 1502 (Fed. Cir. 1988). 

Regarding Claims 2, 3, 14, and 15, as previously mentioned, de Smet 
produces a motion vector for each segment in an image. De Smet , § 2.3. As a 
result, the most important moving areas are determined. Id. at § 3. This is the 
claimed production of motion vectors and at least one motion path. 

Regarding Claims 5 and 17, in Zhang , a clustering in which initialization is 
randomized is described (pg. 11). Since Applicant admits in page 10 of the 19 
January 2010 Appeal Brief that this initialization is an initialization of the 
algorithm as a whole, it inherently contains an initialization of the regression 
function. 

Regarding Claims 7 and 19, insensitivity to initialization is an inherent 
result of the K-Harmonic Means algorithm. Zhang , abstract. 

Regarding Claims 10 and 12, in de Smet , pixels are set as (x, y, t) triples, 
with x and y as spatial coordinates and t as a time coordinate. De Smet , § 2.2. 
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Regarding Claims 11 and 23, de Smet illustrates motion fields. De Smet , 
figs. 3-6. Although these motion fields are not shown as overlaid on the images, the 
examiner takes Official Notice that it was well-known in the art at the time of the 
invention to display a motion field superimposed on an image to provide a visual 
representation of motion vectors. 

Regarding Claims 12 and 24, de Smet illustrates highlighted motion 
segments overlaid on an image. De Smet , figs. 11, 12. 

3. Claims 8, 9, 20, and 21 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over de Smet in view of Zhang as applied to claims 1 and 13 above, 
and further in view of "A Video Segmentation Algorithm for Hierarchical Object 
Representations and its Implementation" (Hermann) . Claims 8, 9, 20, and 21 
disclose using color information to segment techniques, but de Smet only discloses 
"standard watershed techniques" to perform initial segmentation without providing 
details. De Smet . § 2.2. 

Hermann discloses a method for image segmentation to extract objects from a 
moving image. Regarding Claim 8, 9, 20, and 21, after an initial block-matching 
motion estimation, similar to that in de Smet , images are segmented according to 
specific color information, followed by shape analysis, and lastly motion analysis to 
merge regions to determine objects. Herrmann, § II. This color, shape, and motion 
information form the claimed "predetermined criteria". In color analysis, a region is 
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determined as homogeneous if the pixel difference in the region is below a 
threshold. Homogenous, connected areas are determined as "quasi-flat zones". 
These quasi-flat zones are further processed and become the basis for further 
segmentation. Id at § II.B. Then, the color analysis is the claimed step of 
"portioning data according to color". 

De Smet , in combination with Zhang , disclose a majority of the features of 
claims 8, 9, 20, and 21 of the claimed invention, as discussed above, except for color 
segmentation. Herrmann teaches that it was known to segment a moving image 
according to color. Therefore, it would have been obvious to one having ordinary 
skill in the art at the time the invention was made to determine image 
segmentation by color as taught by Herrmann , since Herrmann teaches in p. 205, 1 
3, that color analysis produces the most accurate type of segmentation. 

(10) Response to Argument 

At issue is whether 35 U.S.C. §§ 102(b) and 103(a) place a statutory bar and 
loss of patent rights to an inventor who files a patent application for a specific use of 
a discovered algorithm more than one year after publishing a paper that describes 
the algorithm and contemplates the claimed specific use. 

In October 1999, assignee Hewlett-Packard Co. published "K-Harmonic 
Means - A Data Clustering Algorithm" (the " Zhang " reference), authored by 
Appellant Bin Zhang and two others. This paper proposes a K-Harmonic Means 
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algorithm described as "a center-based clustering algorithm which uses the 
Harmonic Averages of the distances from each data point to the centers as 
components to its performance function". Zhang , abstract. This K-Harmonic Means 
algorithm is designed to improve on a noted fault with a clustering algorithm called 
K-Means algorithm: the K-Means algorithm is dependent on well-placed 
initializations of centers of the different data clusters - the initialized centers, if 
placed poorly, can take a long time, if not indefinitely, to resolve themselves to 
actual centers of data clusters. The K-Harmonic Means algorithm eliminates this 
fault, yielding rapid convergence of the initialized centers to data clusters, even if 
the centers were poorly placed. IcL 

The Zhang reference describes various applications for clustering techniques 
like K-Means and K-Harmonic Means, such as data mining, statistical data 
analysis, and "data compression and vector quantization". Zhang at § 1. The third 
stated application of clustering, data compression and vector quantization, is 
described with respect to the older K-Means algorithm in the de Smet reference, 
which describes performing the K-Means algorithm to segment digital images based 
on a motion vector field. 

The present application is directed toward systems for "performing regression 
clustering using a K-Harmonic Means function to cluster the data points [in an 
image sequence] and to provide motion information regarding the data points" as a 
way to "estimat[e] motion trials in video image sequences". The three independent 
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claims, 1, 13, and 25 disclose, respectively, a data medium storing executable 
instructions to perform the K- Harmonic Means function, a machine that performs 
the K-Harmonic Means function, and a method of performing the K-Harmonic 
Means function. This application was filed on March 17, 2004, nearly 4 1/2 years 
after the Appellant's initial publication disclosing how to perform the K-Harmonic 
Means function. 

During prosecution, the examiner rejected the claimed invention over a 
combination of the de Smet K-Means image data segmentation and the Zhang K- 
Harmonic Means algorithm, with the rationale for substituting the K-Harmonic 
Means algorithm for the K-Means algorithm to take advantage of the known 
improvements described in the Zhang reference of using the K-Harmonic Means 
algorithm over the K-Means algorithm. To the examiner's recollection, Appellant 
has not disclaimed that the K-Harmonic Means algorithm is somehow uniquely not 
suited as an improvement over the K-Means algorithm for the specific claimed 
application of image data segmentation. 

35 U.S.C. § 102 "deals with the two questions of 'novelty' and loss of right". 
In re Bass . 177 U.S.P.Q. 178, 189 (C.C.P.A. 1973). Of the seven sections of § 102, 
the four "prior art" sections, (a), (b), (e), and (g) have relation to § 103 and relevancy 
to "prior art" under § 103. IcL Analysis of the relations between §§ 102(b) and § 103 
includes recalling that Congress placed § 102(b) "as a sort of statute of limitation . . 
. within which an inventor, even though he has made a patentable invention, must 
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act on penalty of loss of right to his patent." In re Foster , 145 U.S.P.Q. 166, 173 
(C.C.P.A. 1965). The one year period starts running based on the availability of the 
invention to the public through the categories of disclosure enumerated in 102(b), 
which include a 'printed publication' anywhere describing the invention". IcL While 
the Foster court recognized that "there appears to be no dispute" that a single 
"printed publication" made publically available over a year causes a loss of patent 
rights to any invention described in that publication ( Id.) , the court asked how this 
analysis relates to the question of whether such a publication renders its contents 
obvious to one of ordinary skill in the relevant art. IcL Reading that the phrase "a 
printed publication" in 102(b) can be interpreted both in the singular and the plural, 
and considering the intent of Congress to make § 102(b) a "time-bar" that predicates 
a loss of right, the Foster court concluded that the § 102(b) loss of rights not only 
includes cases of anticipated inventions, but also cases of obvious inventions. Id. at 
173-174. As such, the loss of rights under § 102(b) analyzed at the time of filing 
supercedes the requirement of § 103 obviousness analyzed at the time the invention 
was made. IcL at 174 (emphasis added). 

In this case, Appellant published a disclosure of "the K-Harmonic Means 
algorithm" in October 1999, and then in March 2004 filed a patent application 
containing, in the body of each independent claim, instructions for performing "a K- 
Harmonic Means" function in the context of clustering image data. Appellant, 
however, does not consider his 1999 K-Harmonic Means publication a barrier of 
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patentability to his 2004 K-Harmonic Means application, but alleges that the 
examiner's mapping of the two identically-named algorithms in the context of image 
data clustering contains "several fundamental incorrect assertions". Appeal Brief at 
Pg. 8. 

Appellant first alleges that the examiner was incorrect in equating the de 
Smet "difference or distance between motion vectors" and the claimed "error". 
Appellant discusses the motion vectors of de Smet , arguing that in that reference, "a 
difference or distance between motion vectors refers to a difference or distance 
between motion vectors for corresponding multiple segments of an image". 
Appellant then states that in the examiner's position that the de Smet motion 
vectors in a motion field are the claimed "motion paths" is erroneous since the 
claimed invention does not disclose calculating differences or distances between 
motion paths but between "data points and corresponding K regression functions 
estimating motion paths". However, first, Applicant does not address why "the 
errors between data points and corresponding ones of the K regression functions 
estimating motion paths" are not or cannot be "motion vectors" in a motion field. 
Appellant's argument that the distance or difference between motion vectors in de 
Smet would represent a distance or difference between motion paths as claimed is 
irrelevant — as Appellant admits in pg. 9 of the Appeal brief, claim 1 is not directed 
to calculating a difference or distance between motion paths, but errors between 
data points and corresponding points of the K regression functions, to estimate 
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these motion paths. The Claim 1 rejection did not discuss differences or distances 
between motion vectors 2 , but rather the distance d(i,l) between data point x(i) and 
corresponding center point m(l) in the Zhang reference. 

Appellant further alleges that the center points of Zhang are not equivalent 
to "the endpoints of the motion vectors" in de Smet or the claimed "motion paths". 
Applicant argues that if the endpoints of Zhang are geometric center positions of 
respective clusters of data points, they cannot be the de Smet motion vector 
endpoints, since "Zhang has nothing to do with providing motion information 
regarding center points". While this may be true for Zhang itself, the rejection is 
not made solely on the Zhang reference, but in a combination between the Zhang 
reference, mainly directed to an abstract algorithm, and de Smet . which discloses 
the use of this type of algorithm in image analysis based on motion vectors. 
Appellant has not shown that the Zhang algorithm is not applicable to image data 
clustering, when the Zhang algorithm is specifically described as being designed as 
an improvement over the K-Means algorithm used for image data clustering in the 
de Smet reference. 

Appellant has not proven that what he meant by "K-Harmonic Means" in 
October 1999 and what he meant by "K-Harmonic Means" in March 2004 are so 

2 Applicant relies on the "Response to Arguments" section of the Final Rejection, which described "a 
difference or distance between motion vectors" as "the claimed 'error'". While Applicant has shown 
that this alternative claim interpretation is faulty, Applicant has not addressed the discussion in the 
claim rejection itself that in the Zhang distance between a data point and a corresponding center 
point in a cluster is patentably distinct between the claimed "error" between a data point and a 
corresponding point in a K-regression function. 
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different as to make the algorithm described by the newer use of this label 
patentably distinct over the older use of this label. Considering the above, the 
October 1999 Zhang reference, in combination with the de Smet reference that 
presents a ready use for the K-Harmonic Means algorithm, is a statutory bar to 
the patentability of the claimed application filed in March 2004. 

Appellant in page 11 of the appeal brief alleges that the rejection of claim 5 
was in error since the Zhang reference does not disclose the claimed random 
initialization of regression functions. However, Zhang in page 11 describes an 
experiment in comparing the performance of the claimed K-Harmonic Means 
algorithm with prior art algorithms, in which each algorithm starts 'from the same 
random initialization". Page 14 illustrates the results with the two prior art 
algorithms and page 15 illustrates the results with the claimed K-Harmonic Means 
algorithm. 

Appellant in pages 11 and 12 presents pro forma arguments on claims 13 and 
25 as parallel in scope to claim 1 and claim 17 as parallel in scope to claim 5. The 
examiner will, likewise, not repeat the above arguments with respect to these 
claims, but only note their applicability as directed to material parallel in scope 
with the claims discussed at length in this brief. 



Application/Control Number: 10/802,428 Page 16 

Art Unit: 2483 

(11) Related Proceeding(s) Appendix 

No decision rendered by a court or the Board is identified by the examiner in 
the Related Appeals and Interferences section of this examiner's answer. 

For at least the above reasons, all claim rejections should be sustained. 
Respectfully submitted, 
/D. N. W./ 

Examiner, Art Unit 2483 
Conferees: 

/JOSEPH USTARIS/ 

Supervisory Patent Examiner, Art Unit 2483 
/Mehrdad Dastouri/ 

Supervisory Patent Examiner, Art Unit 2486 



